In recent years, the use of internet content such as, for example, internet web pages, web advertisement, electronic forms and others, has become widely popular for data processing. Some web pages may include various web pages elements, some being static and others being interactive. These web page elements may correspond to different types and may be associated with a plethora of visual characteristics due to their variety and contrasting nature.
Typically, a web page is defined by a document containing HyperText Markup Language (HTML) code. An HTML document suitable for posting on the internet includes both “content” and “markup.” The content is information which describes a web page's text or other information for display or playback on a computer's monitor, speakers, etc. The markup is information which describes the web page's display characteristics, such as how the content is displayed and how other information can be accessed via the web page.
In order to provide web-based information and services over the internet, the web employs client computers, browser software and servers. A client computer is a computer used by an individual to connect to the internet and access web pages. A browser is a software application, located on a client computer, which requests, via the internet, a web page from a server. After receiving the web page, the browser displays the web page elements on the client computer's display. A server is a computer which stores or hosts web page information, retrieves that information in response to a browser's request, and sends the information, via the internet, to the client computer.
More specifically, the browser software may encompass a rendering engine for rendering the content information with corresponding markup on the display of the client computer. The rendering engine is responsible for displaying the content that the user sees. It communicates with the networking layer of the browser to retrieve the HTML code and other items passed from the server. The rendering is generally done by following the steps of: parsing the HTML, constructing the render tree, layout processing and painting.
The HTML code is a hierarchal structure that begins with an <html> tag, usually contains a <head> and <body> tag, and further includes web page content that corresponds to various web page elements. These web page elements are parsed and turned into a Document Object Model (DOM) tree by the rendering engine. The DOM tree is a structure where each tag is a branch thereof. Style attributes are also passed from the hosting server before being parsed and combined with the DOM tree to create a render tree. The render tree is a structure of visual characteristics such as height/width, style, and ordered in the hierarchy in which they are to be displayed in the browser.
Once the render tree is constructed, the rendering engine recursively goes through the web page elements in the DOM tree and figures out where they should be placed on the display. In other words, the web page elements and their features are mapped to coordinates on the screen based on their respective positions. Each branch of the render tree is drawn out on the display by communicating with the Operating System Interface of the client computer.
Web page elements may correspond to different known visual entities on the web page. For instance, web page elements may correspond to web advertisement, to videos, to interactive objects, to electronic filling forms, etc.